ADX-141: Refactor pipeline/pipelinestep CLI init commands to flag-based inputs by Copilot · Pull Request #1010 · Clarifai/clarifai-python

Copilot · 2026-04-07T05:03:57Z

Interactive click.prompt() calls in pipeline init and pipelinestep init commands made them unusable in CI/CD and scripted environments. This replaces all interactive prompts with CLI flags, aligning with the model CLI interface. When flags are omitted, IDs are now resolved from the global config before falling back to an interactive prompt.

Why

Pipeline commands must work non-interactively in scripts, agents, and CI pipelines
Consistent CLI UX across all Clarifai commands (model CLI already uses flags)
user_id/app_id should be auto-detected from the active login context (~/.clarifai/config.yaml) rather than requiring explicit flags or producing unhelpful placeholder values

How

pipeline init — new flags replace all prompts:

# user_id/app_id auto-detected from global config (~/.clarifai/config.yaml)
clarifai pipeline init

# Fully explicit — fully scriptable
clarifai pipeline init \
  --user_id=my_user --app_id=my_app --pipeline_id=my-pipeline \
  --steps stepA --steps stepB

# Template init with parameter overrides (no prompts)
clarifai pipeline init --template=image-classification \
  --user_id=my_user --app_id=my_app \
  --set model_name=resnet50

pipelinestep init — new flags populate config directly:

clarifai pipelinestep init \
  --user_id=my_user --app_id=my_app --step_id=my-step

Key implementation details:

_init_interactive() renamed to _init_flag_based(); all click.prompt() calls removed from the scaffolding path
_init_from_template() now accepts user_id, app_id, pipeline_id, override_params as parameters; --set key=value replaces template parameter prompts
get_config_template() in pipeline_step_templates.py parameterized with step_id/user_id/app_id; TODO comments only generated when placeholder defaults are used
_DEFAULT_PIPELINE_ID constant extracted to avoid coupling between decorator default and runtime logic
Fixed Python scoping bug: list(steps) was calling the CLI list command (which shadows the built-in); changed to [*steps]
Added resolve_id() utility in clarifai/utils/cli.py: resolves a value from CLI flag → global config current context → interactive prompt. Used for user_id and app_id in both pipeline init and pipelinestep init
--num_steps now uses type=click.IntRange(min=1), rejecting 0 or negative values before any scaffold logic runs
All open(..., 'w') calls in pipelinestep init now include encoding='utf-8' to avoid platform-dependent encoding issues

Tests

All tests updated to use flag-based invocations instead of input= strings and click.prompt mocks
test_init_command_without_template_calls_interactive renamed to test_init_command_without_template_calls_flag_based
Pipeline step tests updated to always supply --user_id/--app_id flags (matching the new resolution behavior)
test_init_with_template_option and test_init_without_template_option updated to assert the full argument list passed to _init_from_template and _init_flag_based (prepared path, template name, user_id, app_id, pipeline_id, override_params/step_names)
141 pipeline-related tests pass

Notes

If --user_id/--app_id are omitted and the global config has no values, the user is prompted interactively — making the command safe in both scripted and interactive shells
--num_steps N is available as a shorthand when step names don't matter (defaults to 2, minimum 1)
step_id in pipelinestep init remains optional; a # TODO: please fill in comment is generated in config.yaml when it is not provided

Copilot

Copilot wasn't able to review any files in this pull request.

…ag-based inputs Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/56634357-d28e-490b-b85d-ec9f1907894b Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/56634357-d28e-490b-b85d-ec9f1907894b Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

clarifai/cli/pipeline.py

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Copilot · 2026-04-07T13:38:49Z

clarifai/cli/pipeline.py

+        # Apply defaults for unset values
+        effective_user_id = user_id or "your_user_id"
+        effective_app_id = app_id or "your_app_id"
+        effective_pipeline_id = (
+            pipeline_id if pipeline_id and pipeline_id != _DEFAULT_PIPELINE_ID else template_name
+        )


In template mode, effective_pipeline_id falls back to template_name whenever pipeline_id == _DEFAULT_PIPELINE_ID. Because the --pipeline_id option always supplies that default, this makes it impossible for a user to intentionally set a template-based pipeline’s ID to "hello-world-pipeline" (explicitly passing that value still gets replaced). Consider making --pipeline_id default to None and resolving defaults inside the template vs non-template branches so an explicitly provided value is always respected.

Copilot · 2026-04-07T13:38:50Z

clarifai/cli/pipeline.py

@@ -309,17 +341,57 @@ def init(pipeline_path, template):
    └── README.md           # Documentation

    PIPELINE_PATH: Path where to create the pipeline project structure. If not specified, the current directory is used by default.
+
+    Examples:
+
+        # Basic initialization with defaults
+        clarifai pipeline init
+
+        # Initialize with explicit IDs and steps
+        clarifai pipeline init --user_id=my_user --app_id=my_app --pipeline_id=my-pipeline --steps stepA --steps stepB
+
+        # Initialize with a specific number of steps
+        clarifai pipeline init --user_id=my_user --app_id=my_app --pipeline_id=my-pipeline --num_steps=3
+
+        # Initialize from a template
+        clarifai pipeline init --template=image-classification --user_id=my_user --app_id=my_app
+
+        # Initialize from a template with custom parameters
+        clarifai pipeline init --template=image-classification --user_id=my_user --app_id=my_app --set model_name=resnet50
    """
    # Common setup logic
    pipeline_path = _prepare_pipeline_path(pipeline_path, template)
    if not pipeline_path:
        return  # Error already shown in _prepare_pipeline_path

+    # Resolve step names: explicit --steps take precedence, then generate from --num_steps
+    if steps:
+        resolved_steps = [*steps]
+    else:
+        default_names = ["stepA", "stepB", "stepC", "stepD", "stepE", "stepF"]
+        resolved_steps = [
+            default_names[i] if i < len(default_names) else f"step{i + 1}"
+            for i in range(num_steps)
+        ]


--num_steps currently accepts 0 or negative values, which produces an empty resolved_steps list and generates a pipeline project with no step directories/configs. Since the generated pipeline config’s step_directories and orchestration spec are meant to reference created step folders, this can lead to a non-functional scaffold. Consider enforcing a minimum of 1 (e.g., type=click.IntRange(min=1) or a runtime validation that errors when num_steps < 1).

tests/test_pipeline_templates.py

Copilot · 2026-04-07T13:38:50Z

clarifai/cli/pipeline_step.py

    else:
        pipeline_step_template = get_pipeline_step_template()
        with open(pipeline_step_py_path, 'w') as f:
            f.write(pipeline_step_template)
        logger.info(f"Created {pipeline_step_py_path}")

    # Create requirements.txt
    requirements_path = os.path.join(pipeline_step_path, "requirements.txt")
    if os.path.exists(requirements_path):
        logger.warning(f"File {requirements_path} already exists, skipping...")
    else:
        requirements_template = get_requirements_template()
        with open(requirements_path, 'w') as f:
            f.write(requirements_template)
        logger.info(f"Created {requirements_path}")

    # Create config.yaml
    config_path = os.path.join(pipeline_step_path, "config.yaml")
    if os.path.exists(config_path):
        logger.warning(f"File {config_path} already exists, skipping...")
    else:
-        config_template = get_config_template()
+        # Pass explicit values; get_config_template uses placeholder defaults for None
+        config_template = get_config_template(
+            **{
+                k: v
+                for k, v in [('step_id', step_id), ('user_id', user_id), ('app_id', app_id)]
+                if v
+            }
+        )
        with open(config_path, 'w') as f:
            f.write(config_template)
        logger.info(f"Created {config_path}")


File writes in pipelinestep init use open(..., 'w') without an explicit encoding. Other CLI init flows in this repo write templates with encoding='utf-8' (e.g., clarifai/cli/pipeline.py:582). To avoid platform-dependent defaults and potential Unicode issues, write these files with an explicit UTF-8 encoding as well.

@copilot apply changes based on:

this feedback

and

this feedback

Done in commit a110b70:

All three open(..., 'w') calls in pipelinestep init now use encoding='utf-8'

--num_steps now uses type=click.IntRange(min=1), so 0 or negative values are rejected with a clear Click error message

Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/e41b21f7-f0ef-47ea-a928-5e04effc1130 Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/4d08f093-dede-4f7d-ba5f-8fbb045ae478 Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

github-actions · 2026-04-07T13:58:49Z

Package	Line Rate	Health
clarifai	45%	❌
clarifai.cli	61%	➖
clarifai.cli.templates	71%	➖
clarifai.cli.templates.toolkits	100%	✔
clarifai.client	65%	➖
clarifai.client.auth	67%	➖
clarifai.constants	100%	✔
clarifai.datasets	100%	✔
clarifai.datasets.export	69%	➖
clarifai.datasets.upload	75%	➖
clarifai.datasets.upload.loaders	37%	❌
clarifai.models	100%	✔
clarifai.rag	0%	❌
clarifai.runners	52%	➖
clarifai.runners.models	60%	➖
clarifai.runners.pipeline_steps	39%	❌
clarifai.runners.pipelines	72%	➖
clarifai.runners.utils	62%	➖
clarifai.runners.utils.data_types	72%	➖
clarifai.schema	100%	✔
clarifai.urls	58%	➖
clarifai.utils	64%	➖
clarifai.utils.evaluation	16%	❌
clarifai.workflows	95%	✔
Summary	60% (11908 / 19769)	➖

Minimum allowed line rate is 50%

… >= 1 Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/dffc4e48-ca95-46f1-9a6c-a918ff4528ee Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

Initial plan

18790d2

Copilot AI assigned Copilot and nitinbhojwani Apr 7, 2026

Copilot AI linked an issue Apr 7, 2026 that may be closed by this pull request

ADX-141 Refactor the clarifai pipeline, pipelinestep, and other pipeline commands to replace interactive prompts with flag-based inputs #1009

Open

Copilot started work on behalf of nitinbhojwani April 7, 2026 05:04 View session

nitinbhojwani requested a review from Copilot April 7, 2026 05:07

Copilot AI reviewed Apr 7, 2026

View reviewed changes

Copilot AI and others added 2 commits April 7, 2026 05:16

Refactor pipeline CLI commands to replace interactive prompts with fl…

1ac6a12

…ag-based inputs Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/56634357-d28e-490b-b85d-ec9f1907894b Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

Address code review feedback: extract constants, simplify config kwargs

427e2b0

Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/56634357-d28e-490b-b85d-ec9f1907894b Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Refactor clarifai pipeline to use flag-based inputs~~ ADX-141: Refactor pipeline/pipelinestep CLI init commands to flag-based inputs Apr 7, 2026

Copilot finished work on behalf of nitinbhojwani April 7, 2026 05:19

Copilot AI requested a review from nitinbhojwani April 7, 2026 05:19

nitinbhojwani requested a review from Copilot April 7, 2026 13:35

Copilot started reviewing on behalf of nitinbhojwani April 7, 2026 13:35 View session

nitinbhojwani reviewed Apr 7, 2026

View reviewed changes

clarifai/cli/pipeline.py Show resolved Hide resolved

Copilot started work on behalf of nitinbhojwani April 7, 2026 13:37 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

Resolve user_id and app_id from global config before prompting

0941f36

Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/e41b21f7-f0ef-47ea-a928-5e04effc1130 Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

Copilot AI requested a review from nitinbhojwani April 7, 2026 13:50

Copilot finished work on behalf of nitinbhojwani April 7, 2026 13:50

Copilot started work on behalf of nitinbhojwani April 7, 2026 13:50 View session

Assert full call arguments in template init routing tests

c83e60f

Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/4d08f093-dede-4f7d-ba5f-8fbb045ae478 Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

Copilot finished work on behalf of nitinbhojwani April 7, 2026 13:55

Copilot started work on behalf of nitinbhojwani April 7, 2026 14:06 View session

Add encoding='utf-8' to pipelinestep file writes; enforce --num_steps…

a110b70

… >= 1 Agent-Logs-Url: https://github.com/Clarifai/clarifai-python/sessions/dffc4e48-ca95-46f1-9a6c-a918ff4528ee Co-authored-by: nitinbhojwani <9331380+nitinbhojwani@users.noreply.github.com>

Copilot finished work on behalf of nitinbhojwani April 7, 2026 14:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ADX-141: Refactor pipeline/pipelinestep CLI init commands to flag-based inputs#1010

ADX-141: Refactor pipeline/pipelinestep CLI init commands to flag-based inputs#1010
Copilot wants to merge 6 commits intomasterfrom
copilot/adx-141-refactor-clarifai-pipeline

Copilot AI commented Apr 7, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

nitinbhojwani Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

github-actions bot commented Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

How

Tests

Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

nitinbhojwani Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Apr 7, 2026 •

edited

Loading